Definitely. I had this issue with Charge 4 to. I live in Sweden(almost always cold) and Cyprus(always warm).
To work around the problem. I check the app on my phone to not expose my arm and device to the cold. Which leads to a bit more regular heart rate. Unfortunately the app crashes my sense(force restarts it) the last 4=5 months. So I have to expose my arm to check on stuff
It is definitely a difference. I can have pulse of 110 an arrive from outside to my home and it jumps about 10 when arrive indoors
It is very annoying It was more predictable when could use my phone to check my stats.
When I in Cyprus. No such issues