TL;DR: Go to GitHub and clone
foggycam to capture unlimited-length Nest video streams to any storage of your choosing without paying for Nest Aware.
VERY IMPORTANT NOTE: What I describe below was put together by analyzing how the Nest apps communicate with the infrastructure. None of these items use official REST APIs (unfortunately, those don’t expose the video stream), therefore can break at any time.
We recently got a Nest cam, and we absolutely love the capabilities it brings to our home. One of the staples of the camera was the capability to record footage and then replay it later. The problem with that is we needed to pay for a subscription, and in my humble opinion, it’s a bit pricey.
There are many workarounds mentioned publicly, that all suggest using third-party software, that require making the camera stream public under the assumption that nobody can guess the URL. For the love of everything, do not do this unless you are OK with all your video being public and open to anyone who wants to see you or your house.
So I thought I’d spend a weekend (notice a pattern with weekend projects?) to figure out the inner workings of the video capture mechanism, and how I can capture video locally without having to bind myself to a paid subscription or making the stream public - the camera is already in my house, it captures the video through my own network, so getting captured static and dynamic images should be relatively painless. This was the day
foggycam was born.
Getting Fundamental Data
My tool belt of choice here was Python and related libraries - I could just write a script that captures everything I need stored, and it will work seamlessly across platforms. My starting point was analyzing the traffic from the Nest Home website.
What I quickly noticed was the fact that the request authentication was mostly done via cookies after the original authorization. However, there were some initialization steps that needed to be taken beforehand.
First and foremost, there is a
POST request done against
https://home.nest.com/session, that initializes the current user session. I formalized that in a simple function:
The payload here is the JSON-ified representation of the Nest username and password. This request mimics that performed through the Nest web app. You might also notice an interesting global variable -
merlin. This is essentially a web request maker, that is preserving cookies as requests are performed.
Remember - after the original auth is performed, calls are not receiving any explicit auth tokens, but rather are reading in domain-specific cookies. When the request is performed, if it is successful, you will get a JSON with detailed user information.
In addition, I am reading in and storing the access token for another future call, that is going to required it
useridI am reading in
initializeSessionis not the same as the user email, but is rather a numeric identifier used internally.
That’s all fine and dandy, however one token that is required to be stored inside the cookie, that is not yet in our possession is
website_2. Particularly, I could not figure out where it’s being generated through the web app because a lot of the requests seem to be already coming with it built-in, so likely some piece of JS code was generating it on the fly.
I noticed that there was a response from
https://home.nest.com/dropcam/api/login that did a
website_2, but I was still hitting authentication issue with the call, even though I passed the required credentials.
After hitting my head against the wall a couple of times, I thought I would double check how the mobile Nest app handles authentication. With a little bit of
mitmproxy help, I’ve noticed that the iOS app was making a different request, to the following URL:
I wonder if I can use that - so I send the exact same payload to it, but to a different URL:
Voila! Just like that, the call succeeded and I managed to get the coveted
website_2 stored in my cookie jar.
Next, I wanted to get some information about the cameras I have registered. That can be obtained with the help of
https://home.nest.com/api/0.1/user/#USERID#/app_launch. Given that I already was in possession of the
userid, I can just substitute that here, and perform user initialization:
One piece of information that I am looking for is the camera ID - when I get the user information, I get a list of objects, such as the geofence, thermostats, etc. I just need the camera, and it appears that the most straightforward way is to find objects that start with
quartz. in their name. The object ID, stripped of the prefix (in this case,
quartz.), is the camera UUID.
NOTE: I tried to write the code in a way that supports multiple cameras, but only have one myself - if you have more than one Nest camera, let me know how it works for you!
We now have everything we need to make sure we can capture images and produce the associated video contnent.
Capturing Images & Producing Video
Nest does not expose the video stream directly - it’s piped through WebSockets, and is DRM-d, therefore without having the key, it’s pointless to even attempt to capture it. That said, Nest does expose an endpoint that gives the image of the current camera state:
Remember, that we already have the camera UUID, and the cookies allow us to
GET anything through this endpoint, as long as we are authorized to do so.
captureImages does just that.
This function conveniently provides a way to store images either in the script folder, or in any custom folder of your choosing, via the
custom_path parameter. So if you want to just dump all content in your Dropbox, OneDrive or Box folder, you can do so by pointing this to a path that syncs to any of the listed (or unlisted) backup services.
NOTE: The tool doesn’t yet support uploading content directly to cloud storage providers. This is on my TODO list, so that you can run the script in the cloud (e.g. inside a container or VM).
In addition to storing images, I want to also combine those in a video - given that we are not dealing with the DRM-d stream, we can just perform multiple requests to get image snapshots and then combine them in a video with the help of ffmpeg:
Once the requests are processed, you will get a
.mp4 file in the folder, generated after fixed intervals - and by that, I mean having a fixed number of buffered images that can be combined in a video. For testing purposes, I set that threshold to be 200, which roughly translates into 8 second video clips - that way it’s uploaded fast and in consumable chunks (~1MB each given current
And just like that, you have local captures of the Nest video without paying for Nest Aware. You can download the tool from GitHub, rename the
_config.json file to
config.json, specify your Nest credentials, and run the script via
Room for Improvement
There are a lot of pieces still missing in this tool, like the ability to upload directly to different cloud providers, checking for token expiration, optimizing the video and image storage and more. I will be working on that in my free time, so stay tuned for updates!
Have any thoughts? Let me know on Twitter!