Timeline was one of those maps features that our users loved, despite it not being super discoverable in the app. A lot of users treated it as a diary and would write pages of personal notes in a textbox box next to their daily trips (for a while there was a bug where that text had no char limit). Sometimes I'd have to debug the spanner rows - it wasn't uncommon to see transcriptions of people's diet and exercise logs. One guy had poems about his wife in there.
Very impressive system that had a big budget in the mid 2010s (mostly built by zurich, iirc). They even hired a bunch of tvcs to walk around movie theaters and scan the wifi ssids, so timeline could show you what movie you saw. It had a photos integration as well that would show the pics you took that day. All sorts of plans for more delightful features like that. I think the value prop was that deeply integrating people's memories made maps a stickier product.
The investment and headcount started getting cut post-pandemic, like everything else at Google. Lots of team churn, not just on Timeline but on hulk and the semantic location service which undergirded it. When I was last there SLS was literally 1 guy who either could not or would not leave. Those services became abandonware, along with all the flumes and postwrite processors responsible for cleaning up the data and enforcing heuristics. Exactly 0 people on the web UI - some of the directories were literally un-reviewable (the code owners had left and no one in geo had MPA-approval). That decay led to a noticeable decline in the accuracy of reported trips. Users weren't happy, angry reports started piling up about inaccurate/missing trips. It was embarassing. Timeline was moved to the back burner and the idea of being a cute time capsule for users no longer aligned with the AI maximalists.
By '22 the investment and headcount was slashed. IMO the ODLH death march was as much about throwing in the towel for Timeline as a product as it was about getting location data off of Google's servers.
I ended up losing nearly 15 years of my Google Location history during the switch to on-device, so if you're interested in doing analyses like this, be sure to back up your data on Takeout before you enable the on-device setting that nemo1618 mentioned. Once that setting is set, the data is no longer available on Takeout, and if the data didn't fully transfer to your device, which is what happened to me and to some others, it's gone: https://www.reddit.com/r/GoogleMaps/comments/1diivt3/megathr...
> Options include keeping your data for three, 18, or 36 months, or indefinitely until you manually delete it.
So, if we Takeout our current data, we can squirrel that away on our own computer.
Also navigate the transition process perfectly, including the above settings, so history -- new history anyway -- will be preserved on Google servers. Will it then be available for decryptable download to the user's computer via Takeout? Or only to a replacement phone?
That encrypted backup isn't available via Takeout, only via the Google Maps app. You can use that backup to load your history to various devices or a replacement phone.
I have my Dad's history as well as mine, and a mapping app into which I can load both. Where the the two tracks coincide, I'm prompted to remember the occasion.
They frame it positively, of course ("now your data lives on your device!") but AFAICT it's all downside. I can't browse my location history on a nice big screen, and (very annoyingly) the app does not let you view your aggregate history over the span of a month or year -- only a single day. Plus, if you lose your phone, you lose all that precious data, unless you configure the app to automatically sync your history to Google's cloud...wait what? Wasn't not doing that the whole point? Just baffling.
Agree with you. I get the intention (a lot of people do find it troubling that all their location data is sent to Google, and appreciate this change), but I don't really care about that for my data, I just want to have everything available so I can analyze it!
Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.
If I had to guess, I'd say they meant they'd prefer a solution where all the location data is stored exclusively on-device, with means to export it to another machine so you can analyze it. I understand the rationale.
The timeline backup is encrypted clientside before upload.
Meaning no casual FBI or police warrant is gonna vacuum it up (at least not from Google, they’ll just go to the cell providers / towers instead as siblings have pointed out).
Obviously yes NSA and CIA and various other nation state attackers will just get it directly off your phone or evil maid you or surveil you in any number of other more traditional ways.
Someone mentioned geofence warrants, i.e. cops/feds asking "Hey Google, tell us which accounts had devices found in these time-space coordinates!", I guess they'd be asking mobile providers to do more logging as an alternative.
A Google account is probably more useful than a SIM card, which might be anonymous or have exchanged hands from the registered buyers, if you as a cop can ask Google to hand over the emails or IPs used by this account, you can find the person's identity and address (if using home IPs subpoena-able by asking the ISP).
I wonder if it's not just American police, imagine this question being asked by Russian FSB, or the "good guys" in the form of the Israeli authorities.
The cell providers actually do erase this data. I tried to subpoena it for a murder suspect to help show he wasn't at the scene, but he left it too late, and Verizon said that they delete their data after 5 years. I don't know the timescales of the other networks.
Not to say that the NSA don't have it all backed up -- I'm sure they do -- but for warrant (i.e. legal process purposes) it probably has a shorter timespan than the Timeline data stuck on Google's infra.
I don't think this is true -- can you explain it? At least for those on US soil, since SCOTUS ruled on Carpenter v. United States, I believe you would need a warrant.
They can already do this via the cellular phone networks, which need your location to provide the service. Yes, they could discard that data once it was no longer needed, but that is subject to subpoena and preservation orders, so it's not that far to a location history DB anyways.
Mobile network location history is a lot more coarse than what Google can provide. Things like interference and reflections limit the precision most cellular networks can provide even when actively probing for a phone's location. With mmWave things are a lot more precise, but reflection can happen even in mmWave configurations.
I don't think Googs knows what free is. They have replaced that word with "ad supported". If TLAs want access to that data, they can just pay for it like everyone else. Gives a whole new meaning to "targeted" ads
There's also a community of people with dedicated "tracking phones" or GPS beacons to record every place they go and store it in apps like https://fogofworld.app
The nice part is that it's not some data silo but that it supports open formats and you can import / export everything very easily.
Looking through that webpage, what I'm not finding are the format of the data, or any tools for exporting it. Is it possible to make use of the data without the FoW app, on a computer?
That's interesting, I found out that Youtube history data can be downloaded as well, I think I can also do a data analysis to look at my youtube viewing history and see what interesting things I can find. I wonder if anyone has already done something similar?
I haven't done it, but I want to.
A very important thing you should know is that only the most recent 5000 videos are kept in the history export, so my advice is to download it frequently.
What I've been doing is regularly getting the .json dumps takeout.google.com and merging them. (for around 4 years now)
I plan on eventually processing that data later to track usual statistics, but also for example my interests over the years (by grouping and then searching for common topics like Minecraft, Haskell, Covid, ...).
Thanks for the heads up! I thought that was the full amount of data.
My idea is to write a purely front-end application that takes a json file and uses a chart to visualize some interesting information by year, such as the most watched channels for the selected year
I wrote a tool a few years ago that could convert your Apple Health data export into a SQLite database. I haven't run it in a while though so there's a chance it doesn't work with the latest file format: https://github.com/dogsheep/healthkit-to-sqlite
I'm sorry, I had completely misunderstood what Show HN means. I thought it was the prefix for indicating the post was made by the poster (what the fine folks at Reddit would call "OC"). Somehow I had missed the https://news.ycombinator.com/showhn.html page.
That is confusing but you can do something similar-ish for a written self-post by writing a comment in the thread saying you're the author, etc. Tends to make the discussion somewhat better too since it seems to reduce tropey first post type comments.
> Update 12/15: This change means that Google will no longer respond to geofence warrants from law enforcement that request information on all devices near a particular incident.
So what? They almost certainly have other sources of location data.
99% of google apps will harvest your personal data by default.
e.g. gboard will send google the name of the app you use their keyboard in every time it's used. the google dialer will send the phone number of every received call to google (for caller and spam ID). Google pay sends every transaction you make to google (including the date, merchant, amount, etc) If you use google photos with backup disabled, they will literally nag you every time you open the app, requesting that you sync their photos - and they use dark patterns to hinder your attempt to decline this.
If you mean "health insurance providers", then we already have that in Switzerland.
My anecdata is when an obese acquaintance walked in to the (for profit) insurance company and asked for "extra" service; the insurer is not obliged to offer that to everyone (whereas there is also a compulsory insurance), simply based on the looks, I guess.
I kinda loathe these insurance providers, however, I can't disagree with some of their policies (lack of exercise induced obesity and smoking is a big no in my book). So indeed, using the "walk enough" information is probably the future, OTOH countries without similar systems (UK, and Eastern Europe comes to my mind) have collapsing public healthcare where they can't filter out self destructing lifestyle, everyone bears the responsibility.
Not being a native speaker I thought the "Kindly enough" conveyed the sarcasm (since I think most people don't associate Google with being kind). I'll try to find a better wording (or maybe make the "Kindly enough" a link pointing the GDPR?)
I actually tried it on a VM a few months back, it was broken in a multitude of fun ways. I think it would be interesting to try and make it work.
There is a very nice blog post about someone explaining how they modified their website's CSS to work with some old browser, maybe Netscape, maybe something else, it was well written and fun to follow, but I cannot for the life of me remember where it was from. Maybe someone will read this comment and know what blog post I'm talking about.
Timeline was one of those maps features that our users loved, despite it not being super discoverable in the app. A lot of users treated it as a diary and would write pages of personal notes in a textbox box next to their daily trips (for a while there was a bug where that text had no char limit). Sometimes I'd have to debug the spanner rows - it wasn't uncommon to see transcriptions of people's diet and exercise logs. One guy had poems about his wife in there.
Very impressive system that had a big budget in the mid 2010s (mostly built by zurich, iirc). They even hired a bunch of tvcs to walk around movie theaters and scan the wifi ssids, so timeline could show you what movie you saw. It had a photos integration as well that would show the pics you took that day. All sorts of plans for more delightful features like that. I think the value prop was that deeply integrating people's memories made maps a stickier product.
The investment and headcount started getting cut post-pandemic, like everything else at Google. Lots of team churn, not just on Timeline but on hulk and the semantic location service which undergirded it. When I was last there SLS was literally 1 guy who either could not or would not leave. Those services became abandonware, along with all the flumes and postwrite processors responsible for cleaning up the data and enforcing heuristics. Exactly 0 people on the web UI - some of the directories were literally un-reviewable (the code owners had left and no one in geo had MPA-approval). That decay led to a noticeable decline in the accuracy of reported trips. Users weren't happy, angry reports started piling up about inaccurate/missing trips. It was embarassing. Timeline was moved to the back burner and the idea of being a cute time capsule for users no longer aligned with the AI maximalists.
By '22 the investment and headcount was slashed. IMO the ODLH death march was as much about throwing in the towel for Timeline as a product as it was about getting location data off of Google's servers.
Also, let’s be honest, as trust in Google evaporated, the idea of you having all that data on our every move just became kinda creepy.
I ended up losing nearly 15 years of my Google Location history during the switch to on-device, so if you're interested in doing analyses like this, be sure to back up your data on Takeout before you enable the on-device setting that nemo1618 mentioned. Once that setting is set, the data is no longer available on Takeout, and if the data didn't fully transfer to your device, which is what happened to me and to some others, it's gone: https://www.reddit.com/r/GoogleMaps/comments/1diivt3/megathr...
> Follow the prompts to set up automatic backups.
> Options include keeping your data for three, 18, or 36 months, or indefinitely until you manually delete it.
So, if we Takeout our current data, we can squirrel that away on our own computer.
Also navigate the transition process perfectly, including the above settings, so history -- new history anyway -- will be preserved on Google servers. Will it then be available for decryptable download to the user's computer via Takeout? Or only to a replacement phone?
That encrypted backup isn't available via Takeout, only via the Google Maps app. You can use that backup to load your history to various devices or a replacement phone.
> I ended up losing nearly 15 years of my Google Location history
genuinely curious—why would you want this?
I have my Dad's history as well as mine, and a mapping app into which I can load both. Where the the two tracks coincide, I'm prompted to remember the occasion.
that's a nice thing to have.
Making analyses like zdimension's, keeping a kind of automated diary, and occasionally looking up a spot I've been but can't quite remember.
I was very annoyed when Google recently announced they were deprecating Timeline for Web: https://support.google.com/maps/answer/14169818?visit_id=638...
They frame it positively, of course ("now your data lives on your device!") but AFAICT it's all downside. I can't browse my location history on a nice big screen, and (very annoyingly) the app does not let you view your aggregate history over the span of a month or year -- only a single day. Plus, if you lose your phone, you lose all that precious data, unless you configure the app to automatically sync your history to Google's cloud...wait what? Wasn't not doing that the whole point? Just baffling.
Agree with you. I get the intention (a lot of people do find it troubling that all their location data is sent to Google, and appreciate this change), but I don't really care about that for my data, I just want to have everything available so I can analyze it!
Then grab it from your device and analyse it. Why should everyone suffer just because you want to be sloppy with your personal data.
Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.
If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
> grab it from your device and analyse it
By what means?
If I had to guess, I'd say they meant they'd prefer a solution where all the location data is stored exclusively on-device, with means to export it to another machine so you can analyze it. I understand the rationale.
I mean, the choice would be nice. Allow people to turn on online sync, keep the data offline, or disable the location recording altogether.
A couple of years ago they redesigned the mobile experience for timeline and since then it’s become so unresponsive it’s basically unusable.
I don't know why they needed to kill the web interface if they still let you 'backup' your location history to the cloud.
The timeline backup is encrypted clientside before upload.
Meaning no casual FBI or police warrant is gonna vacuum it up (at least not from Google, they’ll just go to the cell providers / towers instead as siblings have pointed out).
Obviously yes NSA and CIA and various other nation state attackers will just get it directly off your phone or evil maid you or surveil you in any number of other more traditional ways.
Someone mentioned geofence warrants, i.e. cops/feds asking "Hey Google, tell us which accounts had devices found in these time-space coordinates!", I guess they'd be asking mobile providers to do more logging as an alternative.
A Google account is probably more useful than a SIM card, which might be anonymous or have exchanged hands from the registered buyers, if you as a cop can ask Google to hand over the emails or IPs used by this account, you can find the person's identity and address (if using home IPs subpoena-able by asking the ISP).
I wonder if it's not just American police, imagine this question being asked by Russian FSB, or the "good guys" in the form of the Israeli authorities.
The cell providers actually do erase this data. I tried to subpoena it for a murder suspect to help show he wasn't at the scene, but he left it too late, and Verizon said that they delete their data after 5 years. I don't know the timescales of the other networks.
Not to say that the NSA don't have it all backed up -- I'm sure they do -- but for warrant (i.e. legal process purposes) it probably has a shorter timespan than the Timeline data stuck on Google's infra.
I’m immensely annoyed by it as well. I frequently used that feature to figure out useful, local information.
Consider for a moment that the FBI and CIA can access and search all of the location histories for everyone in one central location without a warrant.
Perhaps Google recognizes what an existential threat to a free society that is.
That is scary. However, being able to see my own data is more important to me in this case.
I don't think this is true -- can you explain it? At least for those on US soil, since SCOTUS ruled on Carpenter v. United States, I believe you would need a warrant.
They can already do this via the cellular phone networks, which need your location to provide the service. Yes, they could discard that data once it was no longer needed, but that is subject to subpoena and preservation orders, so it's not that far to a location history DB anyways.
Mobile network location history is a lot more coarse than what Google can provide. Things like interference and reflections limit the precision most cellular networks can provide even when actively probing for a phone's location. With mmWave things are a lot more precise, but reflection can happen even in mmWave configurations.
I don't think Googs knows what free is. They have replaced that word with "ad supported". If TLAs want access to that data, they can just pay for it like everyone else. Gives a whole new meaning to "targeted" ads
At least CIA can do this already via cell tower logs.
> activityType": "WALKING", > "probability": 97.82699942588806
Hundreds of thousands of these per person, stored forever in a data centre somewhere.
Imagine how much carbon could have been saved by just fixing this to 1-2 decimal places.
It's probably just a float value and they're only rendered as text for the data export that the user requested, don't you think?
I hadn’t considered that possibility but thanks for giving me hope!
https://whatismylocation.org/
There's also a community of people with dedicated "tracking phones" or GPS beacons to record every place they go and store it in apps like https://fogofworld.app
The nice part is that it's not some data silo but that it supports open formats and you can import / export everything very easily.
I don't think I ever understood Douglas Adams's quote about digital watches and going down from the trees more than I have today.
Looking through that webpage, what I'm not finding are the format of the data, or any tools for exporting it. Is it possible to make use of the data without the FoW app, on a computer?
This is neat. I need to find the smallest Android device I can now to keep in my backpack or pocket, that isn't my main phone.
That's interesting, I found out that Youtube history data can be downloaded as well, I think I can also do a data analysis to look at my youtube viewing history and see what interesting things I can find. I wonder if anyone has already done something similar?
I haven't done it, but I want to. A very important thing you should know is that only the most recent 5000 videos are kept in the history export, so my advice is to download it frequently.
What I've been doing is regularly getting the .json dumps takeout.google.com and merging them. (for around 4 years now)
I plan on eventually processing that data later to track usual statistics, but also for example my interests over the years (by grouping and then searching for common topics like Minecraft, Haskell, Covid, ...).
Thanks for the heads up! I thought that was the full amount of data.
My idea is to write a purely front-end application that takes a json file and uses a chart to visualize some interesting information by year, such as the most watched channels for the selected year
It's there something like this for Apple?
I wrote a tool a few years ago that could convert your Apple Health data export into a SQLite database. I haven't run it in a while though so there's a chance it doesn't work with the latest file format: https://github.com/dogsheep/healthkit-to-sqlite
Looks like this shouldn't belong in Show HN as it's a (interesting!) blog post?
I'm sorry, I had completely misunderstood what Show HN means. I thought it was the prefix for indicating the post was made by the poster (what the fine folks at Reddit would call "OC"). Somehow I had missed the https://news.ycombinator.com/showhn.html page.
That is confusing but you can do something similar-ish for a written self-post by writing a comment in the thread saying you're the author, etc. Tends to make the discussion somewhat better too since it seems to reduce tropey first post type comments.
That is correct: https://news.ycombinator.com/showhn.html. I've taken Show HN out of the title now.
Biggest issue with this: No button to turn it off.
Google provides a mechanism to disable location tracking.
https://support.google.com/accounts/answer/3118687?hl=en
https://www.androidpolice.com/how-to-disable-google-location...
I've seen this as a small part in more general articles on increasing privacy in all of your Google settings.
They're also purging server side location data and moving it to devices.
https://support.google.com/maps/answer/14169818
https://blog.google/products/maps/updates-to-location-histor...
https://9to5google.com/2023/12/12/google-location-history-ti...
From the last link, importantly:
> Update 12/15: This change means that Google will no longer respond to geofence warrants from law enforcement that request information on all devices near a particular incident.
https://www.forbes.com/sites/cyrusfarivar/2023/12/14/google-...
https://9to5google.com/2019/04/13/google-android-location-hi...
So what? They almost certainly have other sources of location data.
99% of google apps will harvest your personal data by default.
e.g. gboard will send google the name of the app you use their keyboard in every time it's used. the google dialer will send the phone number of every received call to google (for caller and spam ID). Google pay sends every transaction you make to google (including the date, merchant, amount, etc) If you use google photos with backup disabled, they will literally nag you every time you open the app, requesting that you sync their photos - and they use dark patterns to hinder your attempt to decline this.
There's always been a button to turn it off. I remember making that choice when I got my first Android phone, more than a decade ago.
Not to mention that it has been opt-in for years.
[flagged]
> Google had to be forced to allow this?
When were they forced? Takeout came out in 2011, I don’t remember them being forced to do it
If you mean "health insurance providers", then we already have that in Switzerland.
My anecdata is when an obese acquaintance walked in to the (for profit) insurance company and asked for "extra" service; the insurer is not obliged to offer that to everyone (whereas there is also a compulsory insurance), simply based on the looks, I guess.
I kinda loathe these insurance providers, however, I can't disagree with some of their policies (lack of exercise induced obesity and smoking is a big no in my book). So indeed, using the "walk enough" information is probably the future, OTOH countries without similar systems (UK, and Eastern Europe comes to my mind) have collapsing public healthcare where they can't filter out self destructing lifestyle, everyone bears the responsibility.
Google had a Data Liberation Front before it was forced by regulation to enable bulk, machine-readable data downloading. Takeout is their product.
Not being a native speaker I thought the "Kindly enough" conveyed the sarcasm (since I think most people don't associate Google with being kind). I'll try to find a better wording (or maybe make the "Kindly enough" a link pointing the GDPR?)
> One day, your insurance company may inquire if you have walked enough to be covered. [I think we're past that point.](https://www.nytimes.com/2024/03/11/technology/carmakers-driv...)
> Best viewed with Netscape Communicator 4.
I wonder if the author actually tests the site with Netscape!
I actually tried it on a VM a few months back, it was broken in a multitude of fun ways. I think it would be interesting to try and make it work.
There is a very nice blog post about someone explaining how they modified their website's CSS to work with some old browser, maybe Netscape, maybe something else, it was well written and fun to follow, but I cannot for the life of me remember where it was from. Maybe someone will read this comment and know what blog post I'm talking about.