Some connections to Fitbit server fail because of SSL error

[maksymbodnar]

Yesterday, at about 3:35 PM PST (23:35 UTC) we started to experience an increased number of unsuccessful connections to https://api.fitbit.com.

Now it is roughly 1-2% of failures (about 37,000 calls failed over last 12 hours). 99% of requests are successful.

Before yesterday, we also saw these failed requests, but the number very, very small: about 10 - 20 per 24 hours.

The reason of failures is SSL related:

SSL operation failed with code 1. OpenSSL Error messages:
error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

Here is the debug dump of troubled request:

New TCP connection #7: 10.34.23.67(56688) <-> 104.16.66.50(443)
7 1  0.0225 (0.0225)  C>S  Handshake
      ClientHello
        Version 3.3
        cipher suites
Not enough data. Found 327 bytes (expecting 32767)
Not enough data. Found 64 bytes (expecting 16384)
Not enough data. Found 327 bytes (expecting 32767)
Not enough data. Found 64 bytes (expecting 16384)
        compression methods
                  NULL
7 2  0.0367 (0.0141)  S>C  Handshake
      ServerHello
        Version 3.3
        session_id[32]=
          dc 92 9c 8e 2d 19 15 f7 21 90 ed 2f 1a f4 c9 2c
          61 3b 5f 5e 75 8f 5c cc ba 63 b8 a1 31 01 c3 aa
        compressionMethod                   NULL
7 3  0.0368 (0.0000)  S>C  Handshake
      Certificate
7 4  0.0368 (0.0000)  S>C  Handshake
      ServerKeyExchange
7 5    0.0368   (0.0000)    S>C    Handshake
        ServerHelloDone
7 6    0.0410   (0.0042)    C>S    Handshake
        ClientKeyExchange
7 7    0.0410   (0.0000)    C>S    ChangeCipherSpec
7 8    0.0410   (0.0000)    C>S      Handshake
7 9    0.0455   (0.0045)    S>C    ChangeCipherSpec
7 10   0.0455   (0.0000)    S>C      Handshake
7 11   0.0459   (0.0003)    C>S      application_data
6 12   0.2579   (0.2239)    S>C      application_data
6 13   0.2580   (0.0001)    S>C      application_data
  6      0.2580   (0.0000)    S>C    TCP FIN
6 14   0.2585   (0.0004)    C>S      Alert
  6      0.2604   (0.0018)    C>S    TCP FIN
7 12   0.1927   (0.1468)    S>C      application_data
7 13   0.1927   (0.0000)    S>C      application_data
  7      0.1927   (0.0000)    S>C    TCP FIN
7 14   0.1931   (0.0003)    C>S      Alert
  7      0.1951   (0.0020)    C>S    TCP FIN 

I will appreciate any help or suggestion about how to solve the problem.

Thanks,

Maksym Bodnar

[maksymbodnar]

Wow, that was fast.

I just started to type, that I posted the question to the wrong board, but you already moved it.

I apologize for that, 

thanks.

[iwantausername]

Looks like the issue with with 1 of the 2 Cloudflare IPs servicing api.fitbit.com. 104.16.65.50 looks fine, but with some https requests to 104.16.66.50, the server is not returning a certificate. This explains the failing SSL/TLS handshake.

$ host api.fitbit.com
api.fitbit.com is an alias for api.fitbit.com.cdn.cloudflare.net.
api.fitbit.com.cdn.cloudflare.net has address 104.16.66.50
api.fitbit.com.cdn.cloudflare.net has address 104.16.65.50

I made 1500 consecutive requests to 104.16.65.50 with this command:

echo "Q" | openssl s_client -connect 104.16.65.50:443 -servername api.fitbit.com -tls1_2

Result: very single one returned a valid certificate and successfully negotiated a TLSv1.2 connection.

I made 1000 consecutive requests to 104.16.66.50 with the same command:

echo "Q" | openssl s_client -connect 104.16.66.50:443 -servername api.fitbit.com -tls1_2

Result: 61 out of the 500 requests (6.1%) failed with the following message:

CONNECTED(00000003)
---
no peer certificate available---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
Start Time: 1548720878
Timeout : 7200 (sec)
Verify return code: 0 (ok)

Note that Session-ID and Master-ID are also normally present, except for with these failing requests where there is no certificate sent from the server. 

[Gordon-C]

Hi @iwantausername, are you still receiving these error messages?

[iwantausername]

@Gordon-C looks like the error is gone. Just completed 3500 requests to each IP without error.