Confusing specs for verifying Subscriptions X-Fitbit-Signature

[wbalex]

Looking at the Security -> X-Fitbit-Signature Header section of:

https://dev.fitbit.com/build/reference/web-api/subscriptions/

> The signature is first BASE64 encoded and then URL encoded, as with the OAuth signature.

I am unable to determine what specific type of URL encoding "URL encoded" is referring to.

Assuming it refers to the "Percent-Encoding"  section of RFC3986: https://tools.ietf.org/html/rfc3986#section-2.1

The spec differentiates with how to properly percent-encode different parts of the URL. Different parts of the URL include one of two different sets of reserved characters (denoted as "gen-delims" and "sub-delims" in the spec).

Popular URL encoding libraries tend to offer separate functions for encoding the part of the URL prior to the query params vs. each individual query param. 

- Ruby Addressable::URI has encode vs form_encode.
- JavaScript has encodeURI() vs encodeURIComponent()

In the context of encoding URLs, the above functions are intuitive to understand. If the thing I am encoding is a form parameter, I should use the "component" or "form" functions, otherwise I should use the broader "encode" functions.  (Notably, the ruby URI.escape library seems to have been removed from Ruby because it did not properly differentiate these use cases.) 

Coming back to the "X-Fitbit-Signature" header: since the Fitbit spec has adopted URI encoding for use in a HTTP header, it is unclear whether the "url" vs. "url component" version of the encoding functions should be used.

Looking at requests from FitBit, I am noticing web hook requests include values like this (some characters changed):

j2gj1aYBacgPCFN19gkNCpPIlbE=

Notably, the incoming raw values have the "=" character (presumably the base64 padding), and sometimes a "+" character. To me, this implies that the value was URL encoded using the "url" variant, not the "url" component variant, otherwise the "=" and "+" characters would have been escaped due to being reserved characters:

# URL encoding

[2] pry(main)> Addressable::URI.encode("j2gj1aYBacgPCFN19gkNCpPIlbE=")

=> "j2gj1aYBacgPCFN19gkNCpPIlbE="

# URL component encoding, = becomes %3D

[4] pry(main)> Addressable::URI.form_encode(foo: "j2gj1aYBacgPCFN19gkNCpPIlbE=")

=> "foo=j2gj1aYBacgPCFN19gkNCpPIlbE%3D"

This leads me to wonder why URL encoding is necessary to begin with, as the only characters in base64 other than alphanumeric chars are "+", "/" and "=", based on the table in RFC4648.

It appears other people have removed the URL encoding as a way to make this work:

https://community.fitbit.com/t5/Web-API-Development/Verifying-X-Fitbit-Signature-Header-in-php/m-p/1...

https://community.fitbit.com/t5/Web-API-Development/Subscription-Notification-signature-validation/m...

Interestingly, I had missed that the second link above is a reply authored by what appears to be a FitBit employee. 

If URL encoding is not necessary here, can the documentation be updated to avoid further confusion?

[JohnFitbit]

Hi @wbalex,

Welcome to the forums, and I apologize for the delayed response.

After checking in with the team, it appears that we have an error in our documentation and that the X-Fitbit-Signature header is not URL-encoded. Thank you for pointing this out. We've created a ticket to update the documentation for this section.

Let me know if you have any other questions.