Toggle Switch with ColorSelect

[jomis003]

I need a fresh set of eyes on something that banging my head with and the answer seems to be in sight but just out of grasp. 

I am working with a toggle switch in settings that when set to true, it displays the colorSelect.

const colorSet = [
 {color: "black"},
 {color: "grey"},
 {color: "lightgrey"},
 {color: "beige"},
 {color: "whitesmoke"},
 {color: "white"},
];

<Toggle
    settingsKey="toggle"
    label="Set Toggle"
/>
    { JSON.parse(props.settings.toggle || 'false') && 
     <ColorSelect settingsKey="colorToggle" 
      colors={colorSet} 
     /> } 

What is eluding me is when the toggle is set to false and the colorSelect collapses, that the setting is removed and set back to default. 

I have tried implementing the props.settingsStorage.clear("colorToggle") but this clears all of the settings instead of just this one. 

Any help is appreciated.